Inspection processing system, inspection processing method, and inspection processing program

ABSTRACT

An inspection processing system identifies a current position, obtains an inspection result of an inspection subject of a structure on the current position, and records the inspection result in association with the position of the inspection subject in a three-dimensional model of the structure. The inspection processing system further identifies a user who views the three-dimensional model and a current position, and outputs, to an output unit, an inspection result in which the user is recorded in an inspection information memory as a related person of the inspection subject. The inspection result is attached on a virtual image according to a three-dimensional model corresponding to the current position.

BACKGROUND

The present disclosure relates to an inspection processing system, aninspection processing method, and an inspection processing program thatassist inspection of a structure.

In a site of building, civil engineering, or the like, situations beforeand after construction working or situations during construction workingare photographed to check or inspect the situations in some cases.Japanese Laid-Open Patent Publication No. 2014-35341 discloses atechnique for measuring the installment precision of a building elementsuch as a column. In this technique, markers are arranged in advance onmultiple predetermined points of the column to be located at apredetermined position. After installation, the markers are photographedwith a stereo camera. The positions of the markers are extracted fromthe photographed images. Then, based on the positions of the markers,installation precision in errors or the like in the actual installationposition of the column relative to the designed installation position ofthe column are computed.

Further, Japanese Laid-Open Patent Publication No. 2016-3981 disclosesan assembled rebar inspection system for inspecting assembled rebars. Inthis technique, if information such as the diameters, number, andpitches of reinforcing bars (rebars) subject to inspection is input, theassembled rebar inspection system measures the diameters, number,pitches, and the like of the rebars subject to inspection based on thephotographed images of the rebars subject to inspection. Then, theassembled rebar inspection system determines whether or not the inputinformation of the rebars subject to inspection matches the measurementinformation of the rebars subject to inspection measured based on thephotographed images.

In addition, Japanese Laid-Open Patent Publication No. 2017-45404discloses an image management system for managing photographed images ofa site. In this technique, a control unit of a tablet terminal executesa process for photographing a construction work record. Then, thecontrol unit records the photographed images in association with thecurrent position, generates photographed image information including thecurrent position in relation to the photographed images, and records thephotographed image information. If the photographing point ispreregistered, the control unit records the coordinates of the point inassociation with the photographed images and adjusts positioninformation of the photographed images on the photographing point fromthe previous time to the present time.

Although the technique of Japanese Laid-Open Patent Publication No.2014-35341 allows for installation precision measurement with a stereocamera, it is not intended for sharing of information about theinspection result. Further, Japanese Laid-Open Patent Publication No.2016-3981 is not intended for sharing of information about theinspection result using images. In addition, the technique of JapaneseLaid-Open Patent Publication No. 2017-45404 allows a photographed imageto remain on a predetermined point but does not increase the efficiencyof inspection by means of information sharing. This is becauseinspection is performed in the same manner as the conventional one.

SUMMARY

It is an object of the present disclosure to efficiently inspect astructure.

To achieve the above object, one aspect of the present disclosureprovides an inspection processing system including circuitry. Thecircuitry is configured to execute a registration process to identify acurrent position, obtain an inspection result of an inspection subjectof a structure on the current position, and record the inspection resultin association with the position of the inspection subject in athree-dimensional model of the structure. The circuitry is furtherconfigured to execute a displaying process to identify a user who viewsthe three-dimensional model and a current position, and output, to anoutput unit, an inspection result in which the user is recorded in aninspection information memory as a related person of the inspectionsubject. The inspection result is attached on a virtual image accordingto a three-dimensional model corresponding to the current position.

Another aspect provides a method for assisting inspection using aninspection processing system. The inspection processing system includescircuitry. The circuitry executes a registration process to identify acurrent position, obtain an inspection result of an inspection subjectof a structure on the current position, and record the inspection resultin association with the position of the inspection subject in athree-dimensional model of the structure. The circuitry further executesa displaying process to identify a user who views the three-dimensionalmodel and a current position, and output, to an output unit, aninspection result in which the user is recorded in an inspectioninformation memory as a related person of the inspection subject. Theinspection result is attached on a virtual image according to athree-dimensional model corresponding to the current position.

A further aspect provides a non-transitory computer-readable medium thatstores a program thereon. The program, when executed by circuitry of aninspection processing system, causes the circuitry to execute aregistration process to identify a current position, obtain aninspection result of an inspection subject of a structure on the currentposition, and record the inspection result in association with theposition of the inspection subject in a three-dimensional model of thestructure. The program further causes the circuitry to execute adisplaying process to identify a user who views the three-dimensionalmodel and a current position, and output, to an output unit, aninspection result in which the user is recorded in an inspectioninformation memory as a related person of the inspection subject. Theinspection result is attached on a virtual image according to athree-dimensional model corresponding to the current position.

Other aspects and advantages of the present disclosure will becomeapparent from the following description, taken in conjunction with theaccompanying drawings, illustrating by way of example the principles ofthe disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure, together with objects and advantages thereof, may bestbe understood by reference to the following description of the presentlypreferred embodiments together with the accompanying drawings in which:

FIG. 1 is a diagram illustrating a system of the present embodiment;

FIGS. 2A to 2D are diagrams illustrating data recorded in a BIMinformation memory, a participant information memory, a delivery rulememory, and an inspection information memory of the system shown in FIG.1, respectively;

FIGS. 3 and 4 are flowcharts illustrating the procedures of the systemshown in FIG. 1, respectively;

FIG. 5 shows pictures illustrating output images of the system shown inFIG. 1;

FIG. 6A is a flowchart illustrating the procedures in an anchoragelength measurement assistance process of the system shown in FIG. 1;

FIG. 6B is a picture illustrating a method for calculating an anchoragelength in the anchorage length measurement assistance process of thesystem shown in FIG. 1;

FIG. 7A is a flowchart illustrating the procedures for a measurementassistance process of a pitch and rebar diameter of the system shown inFIG. 1;

FIGS. 7B to 7E are pictures respectively illustrating point group data,a pitch measurement frame, a pitch calculation method, and a rebardiameter calculation method in the measurement assistance process of thepitch and rebar diameter of the system shown in FIG. 1;

FIG. 8A is a diagram illustrating an output image having a smalldifference in a further embodiment; and

FIG. 8B is a diagram illustrating an output image having a largedifference in a further embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

An inspection processing system, an inspection processing method, and aninspection processing program according to one embodiment will now bedescribed with reference to FIGS. 1 to 7E. In the description of thepresent embodiment, the inspection system assists inspection of astructure (for example, assembled rebars) at a construction work site ofbuilding work. In this case, two-dimensional images (virtual images)generated through building information modeling (BIM) and photographedimages (structure images) at a site are used to assist inspection andshare information about the inspection result among participants.

FIG. 1 shows a management server 10 and a user terminal 20 used toperform assembled rebar inspection of a structure.

The management server 10 is a computer system that manages various typesof data used at the user terminal 20. The management server 10 includesa BIM information memory 12, a participant information memory 13, adelivery rule memory 14, and an inspection information memory 15. Datarecorded in the BIM information memory 12, the participant informationmemory 13, the delivery rule memory 14, and the inspection informationmemory 15 are the same as data recorded in a BIM information memory 22,a participant information memory 23, a delivery rule memory 24, and aninspection information memory 25 of the user terminal 20, which will bedescribed later. Thus, each data will be described in the description ofthe BIM information memory 22, the participant information memory 23,the delivery rule memory 24, and the inspection information memory 25.

The user terminal 20 is a computer terminal that assists inspection of astructure using images photographed at a construction work site. Theuser terminal 20 includes a control unit 21, the BIM information memory22, the participant information memory 23, the delivery rule memory 24,the inspection information memory 25, a touch panel display 26, aninertia measurement unit 27, and a camera 28.

The touch panel display 26 functions as an input unit to which varioustypes of information are input, and functions as an output unit thatoutputs various types of information are output. As long as the inputunit and the output unit are hardware to which information can be inputand output, the input unit and the output unit are not limited to thetouch panel display 26.

The inertia measurement unit 27 functions as a position identificationunit that calculates a movement distance or movement direction byobtaining three-axis acceleration.

The camera 28 functions as a photographing unit that photographs anobject. In the present embodiment, a stereo camera that simultaneouslyphotographs a subject from different directions to obtain information ina depthwise direction of the directions is used. The distance to anobject and point group data for a three-dimensional shape are obtainedusing the camera 28. As long as the photographing unit is hardware thatphotographs an object, the photographing unit is not limited to a stereocamera.

The control unit 21 functions as a control means including a CPU, a RAM,a ROM, and the like and performs processes (processes including, forexample, user authentication stage, position identification stage, imagegeneration stage, image output stage, inspection assistance stage,measurement assistance stage, and delivery processing stage), which willbe described later. By executing an inspection processing program forthe processes, the control unit 21 functions as a user authenticationunit 210, a position identification unit 211, an image generation unit212, an image output unit 213, an inspection assistance unit 214, ameasurement assistance unit 215, a delivery processing unit 216, and thelike.

The user authentication unit 210 executes a process for identifying auser.

The position identification unit 211 executes a process for identifyingthe current place. In the present embodiment, the inertia measurementunit 27 is used to calculate a movement distance or movement direction.The position identification unit 211 identifies the current place basedon the movement distance and the movement direction from the initialposition.

The image generation unit 212 executes a process for generating atwo-dimensional virtual image from a three-dimensional model. In thepresent embodiment, a two-dimensional image (BIM image) is generatedfrom a three-dimensional model of a structure recorded in the BIMinformation memory 22 (described later). The viewpoint of thetwo-dimensional image corresponds to the case of viewing thephotographing direction of the camera 28 from the current position inthe virtual space (BIM space).

The image output unit 213 executes a process for displaying a BIM imagewhere the BIM image is superimposed on an image that is photographed bythe camera 28 and output to the touch panel display 26.

The inspection assistance unit 214 executes a process for assistinginspection of inspection items recorded in the inspection informationmemory 25.

The measurement assistance unit 215 executes a process for assistingmeasurement that is necessary for the inspection items based on aphotographed image. The measurement assistance unit 215 assistsmeasurement of, for example, anchorage length, rebar pitch, and rebardiameter.

The delivery processing unit 216 executes a process for delivering aninspection result based on a delivery rule.

As shown in FIG. 2A, the BIM information memory 22 (model memory)records BIM management data 220 related to a three-dimensional model ofa structure subject to inspection generated through the BIM technique.If a three-dimensional model generated when a structure is designed isdownloaded from the management server 10, the BIM management data 220 isrecorded. The BIM management data 220 includes data related to astructure ID, floor, and three-dimensional model.

Data related to an identifier that identifies a structure is recorded ina structure ID data region.

Data related to an identifier that identifies a floor of the structureis recorded in a floor data region.

A three-dimensional model arranged in virtual space (BIM space) of thefloor is recorded in a three-dimensional model data region. Thethree-dimensional model includes multiple element models. For example,with regard to assembled rebars, a three-dimensional model of a rebararranged on the floor (BIM space) is recorded. The coordinates of aninitial position marker attached to an inspection start position arerecorded in the three-dimensional model.

Each three-dimensional model keeps attribute information. The attributeinformation includes information related to an element ID, element name,element specification, and the like. The element ID is an identifierthat identifies an element. The element name is the name of an element(for example, rebar). The element specification is the specification of,for example, material and size (anchorage length, rebar pitch, rebardiameter, and the like) of an element.

As shown in FIG. 2B, the participant information memory 23 recordsparticipant management data 230 related to participants involved in thestructure building. The participant management data 230 is recorded ifinformation related to a manager or constructor of the structure isregistered to the participant information memory 23 and the informationis downloaded from the management server 10. The participant managementdata 230 includes a construction company record 231, a construction workgroup record 232, and a construction work supervisor record 233.

The construction company record 231 includes data related to an elementID, a construction company ID, and contact information.

Data related to an identifier that identifies an element constituting astructure is recorded in an element ID data region.

Data related to an identifier that identifies the construction companythat performs construction using the element (constructor information)is recorded in a construction company ID data region.

Data related to contact information of the construction company isrecorded in a contact information data region.

The construction work group record 232 includes data related to aconstruction work group ID and contact information.

Data related to an identifier that identifies a construction work groupsupervising the construction is recorded in a construction work group IDdata region.

Data related to contact information of the construction work group isrecorded in a contact information data region.

The construction work supervisor record 233 includes data related to aconstruction work supervisor ID and contact information.

Data related to an identifier that identifies a supervisor of theconstruction work site is recorded in a construction work supervisor IDdata region.

Data related to contact information (for example, email address) of thesupervisor of the construction work site is recorded in the contactinformation data region.

As shown in FIG. 2C, delivery rule data 240 related to a method fordetermining a delivery destination that should be notified of aninspection result and delivery destination management data 241 arerecorded in the delivery rule memory 24. The delivery rule data 240 isrecorded if a determined delivery rule and delivery destination areregistered to the delivery rule memory 14 and downloaded from themanagement server 10.

The delivery rule data 240 includes data related to a delivery level anda delivery destination pattern.

A value that identifies an information sharing level identifying asubject who receives an inspection result is recorded in a deliverylevel data region. The delivery level is calculated from importance andurgency designated by a person in charge of inspection. For example, theimportance and urgency are each designated as 1 to 3. In this case, asthe importance or urgency becomes higher, a higher value is designated.A value calculated by multiplying the importance and urgency (1 to 9) isused for the delivery level.

Data related to a delivery destination corresponding to a delivery level(for example, construction company ID, construction work group ID, andconstruction work supervisor) is recorded in the delivery destinationpattern. For example, if the delivery level is 1 to 2, which is a normallevel, the inspection result is delivered to the construction companyand the construction work group related to construction. If the deliverylevel is 3 to 5, the inspection result is delivered to the constructionwork supervisor in addition to the delivery destinations of the normallevel. If the delivery level is 6 to 9, the inspection result isdelivered to all the construction work groups and the site manager inaddition to the delivery destinations of the normal level. Further, ifthe delivery level is greater than or equal to an urgency referencevalue, urgency delivery, in which the inspection result isinstantaneously delivered to all the participants, is performed.

The delivery destination management data 241 includes data related to aninspection type and delivery destination.

Data related to the type of inspection of a structure is recorded in aninspection type data region.

Data related to the delivery destination that should be notified of, inthe delivery type data, an inspection result through normal delivery isrecorded in a delivery destination data region.

As shown in FIG. 2D, the inspection information memory 25 records aninspection management record 250 related to an inspection result of astructure subject to inspection. The inspection management record 250 isregistered to the inspection information memory 15 prior to inspectionand recorded in synchronization with the inspection information memory15 of the management server 10. Thus, when inspection is performed, theinspection information memory 15 of the management server 10 is updatedas the inspection information memory 25 of the user terminal 20 isupdated. The inspection management record 250 includes data related toinspection ID, inspection type, floor, element ID, element name,three-dimensional coordinates, inspection items, design value,measurement length, related construction work group ID, relevant companyID, inspection result, completion of inspection/dealing, related file,remarks, and delivery destination ID.

Data related to an identifier that identifies each inspection isrecorded in an inspection ID data region.

Data related to the type of the inspection is recorded in an inspectiontype data region.

Data related to an identifier that identifies a floor in a structure isrecorded in a floor data region.

Data related to an identifier that identifies an element subject toinspection is recorded in an element ID data region.

Data related to the name of an inspection subject is recorded in anelement name data region.

Data related to the coordinates that identify an indication position inthe inspection result is recorded in a three-dimensional coordinate dataregion.

Data related to an identifier that identifies an inspection item isrecorded in an inspection item data region.

Data related to a design value of the inspection item is recorded in adesign value data region.

Data related to a value that has been actually measured in theinspection item is recorded in a measurement value data region.

Data related to an identifier that identifies a construction work groupin charge of the inspection item is recorded in a related constructionwork group ID data region.

Data related to an identifier that identifies the construction companyor the like of the element is recorded in a relevant company ID dataregion.

Data related to an inspection determination (pass/fail) with inspectionis recorded in an inspection result data region. Further, data relatedto importance and urgency when the inspection item needs to be dealtwith is recorded in the inspection result data region.

A flag that identifies a dealing situation of the inspection item isrecorded in an inspection/dealing complete data region.

Data related to a link destination of data related to inspection isrecorded in a related file data region. For the related file, athree-dimensional model and writing of the structure and an image(two-dimensional image, a stereo image, or three-dimensional point groupdata) indicating an inspection situation or patrol situation can beused.

Data related to a comment such as an indication matter given by a personin charge of inspection is recorded in a remarks data region.

An identifier that identifies a participant delivering the inspectionresult is recorded in a delivery destination ID data region.

The procedures for inspecting a structure with the user terminal 20 willnow be described with reference to FIGS. 3 to 7E.

Inspection Assistance Process

The inspection assistance process will now be described with referenceto FIG. 3. This process is executed if inspection is performed using theuser terminal 20. First, a person in charge of inspection moves to aposition where the person can photograph the initial position markerattached to a structure. In the user terminal 20, the inspectionprocessing program is activated.

In this case, the control unit 21 of the user terminal 20 executes auser authentication process (step S1-1). More specifically, the userauthentication unit 210 of the control unit 21 outputs a login view towhich user information is input. The user authentication unit 210identifies a person in charge of inspection (for example, person incharge at construction work group) based on a user ID input to the loginview.

Next, the control unit 21 executes a process for identifying the initialposition (step S1-2). More specifically, the position identificationunit 211 of the control unit 21 outputs a photographing instruction of amarker to the touch panel display 26. The person in charge of inspectionphotographs the initial position marker using the camera 28 of the userterminal 20. In this case, the position identification unit 211identifies the initial position of the user terminal 20 (person incharge of inspection) in the virtual space based on the distance anddirection to the initial position marker.

The person in charge of inspection patrols the site for the purpose ofinspection carrying the user terminal 20.

In this case, the control unit 21 executes a process for identifying thecurrent place (step S1-3). More specifically, the positionidentification unit 211 of the control unit 21 obtains accelerationinformation from the inertia measurement unit 27 and calculates themovement direction and movement distance based on the accelerationinformation. The position identification unit 211 adds the movementdistance in the movement direction with respect to the coordinates ofthe initial position and identifies the current position.

Then, the control unit 21 executes a superimposition displaying processof a BIM model (step S1-4). More specifically, the image generation unit212 of the control unit 21 identifies the orientation of the userterminal 20 based on the acceleration information obtained from theinertia measurement unit 27. Subsequently, in the BIM space, the imagegeneration unit 212 arranges a viewpoint on the current place of thecamera 28 of the user terminal 20, identifies a visual range based onthe orientation of the user terminal 20, and generates a two-dimensionalimage (BIM image) of an element subject to inspection. For the BIMimage, a virtual image having a predetermined transmittance is used.

Next, the image output unit 213 outputs an image photographed with thecamera 28 to the touch panel display 26. The image output unit 213superimposes, on the photographed image, a two-dimensional imagegenerated by the image generation unit 212.

In this case, as shown in FIG. 5, a BIM image 510 corresponding to aphotographed image 500 is generated. The image output unit 213 generatesan output image 520, in which the BIM image 510 is superimposed on thephotographed image 500, and outputs the output image 520 to the touchpanel display 26.

Afterwards, the control unit 21 executes a process for determiningwhether or not an element is subject to inspection (step S1-5). Morespecifically, the inspection assistance unit 214 of the control unit 21uses attribute information of a three-dimensional model to identify anelement included in a two-dimensional image. The inspection assistanceunit 214 determines that the element is subject to inspection if theelement ID is recorded in the inspection information memory 25.

If the control unit 21 determines that the element is subject toinspection (“YES” in step S1-5), the control unit 21 executes ameasurement assistance process (step S1-6). More specifically, theinspection assistance unit 214 outputs a message indicating that aninspection subject is included to the touch panel display 26.Additionally, as described below, the measurement assistance unit 215executes a measurement assistance process for measuring the rebardiameter, pitch, anchorage length, and the like of the inspectionsubject (FIGS. 6A, 6B, and 7A to 7E).

If the control unit 21 determines that an element subject to inspectionis not included and that the element is not subject to inspection (“NO”in step S1-5), the control unit 21 skips the measurement assistanceprocess (step S1-6).

Then, the control unit 21 executes a process for determining whether ornot an inspection result has been input (step S1-7). More specifically,if a person in charge of inspection inputs an inspection result, theperson inputs a display instruction of an information registration view.In this case, the inspection assistance unit 214 determines that theinspection result has been input and causes the touch panel display 26to display the information registration view. If shared information ofsome indication matters or the like is input separately from inspection,the information registration view may also be used.

If the control unit 21 determines that the inspection result has beeninput (“YES” in step S1-7), the control unit 21 executes a process forrecording the inspection result (step S1-8). More specifically, theperson in charge of inspection inputs the measurement value and theinspection result (pass or fail, importance and urgency) to theinformation registration view. Further, an image in which a situation isphotographed is generated, and a saving destination is input. If theinspection assistance unit 214 obtains a saving instruction of the inputinformation, the inspection assistance unit 214 generates the inspectionmanagement record 250 in which the measurement value, the inspectionresult, and the three-dimensional coordinates of the current place inthe BIM space are recorded and then records the inspection managementrecord 250 in the inspection information memory 25. In addition, ifwriting is performed on the three-dimensional model at a site or if asituation of the site is photographed using the camera 28, theinspection assistance unit 214 records the writing information and thesaving destination information of a photographed image (stereo image,three-dimensional point group data, or the like) in the related filedata region of the inspection management record 250. Further, if theinspection assistance unit 214 obtains an input instruction of a commenton the information registration view, the inspection assistance unit 214records the input instruction in the remarks data region of theinspection management record 250.

Next, the control unit 21 executes a process for determining whether ornot there is urgency (steps S1-9). More specifically, the deliveryprocessing unit 216 of the control unit 21 calculates a delivery levelbased on the importance and urgency recorded in the inspectionmanagement record 250. If the delivery level is greater than or equal tothe urgency reference value recorded in the delivery rule data 240 ofthe delivery rule memory 24, the delivery processing unit 216 determinesthat there is urgency.

If the control unit 21 determines that there is urgency (“YES” in stepS1-9), the control unit 21 executes an urgent delivery process (stepS1-10). More specifically, the delivery processing unit 216 identifies adelivery destination corresponding to a delivery level, obtains thecontact information of all the participants from the participantinformation memory 23, and delivers an urgent message.

If the control unit 21 determines that there is no urgency (“NO” in stepS1-9), the control unit 21 skips the urgent delivery process (stepS1-10).

Subsequently, the control unit 21 executes a process for determiningwhether or not the inspection has ended (step S1-11). More specifically,if an end input is performed on the touch panel display 26, theinspection assistance unit 214 determines that the inspection has ended.

If the control unit 21 determines that the inspection has not ended(“NO” in step S1-11), the control unit 21 repeats the processessubsequent to the process for identifying the current place (step S1-3).

If the control unit 21 determines that the inspection has ended (“YES”in step S1-11), the control unit 21 executes a process for deliveringthe inspection result (step S1-12). More specifically, the deliveryprocessing unit 216 of the control unit 21 identifies an inspectionresult that requires notification based on the inspection result of theinspection management record 250 recorded in the inspection informationmemory 25. The delivery processing unit 216 calculates a delivery levelbased on the importance and urgency recorded in the inspectionmanagement record 250. Next, the delivery processing unit 216 uses thedelivery rule data 240 of the delivery rule memory 24 to identify thedelivery destination pattern corresponding to the delivery level. Thedelivery processing unit 216 records, in the delivery destination IDdata region recorded in the inspection management record 250, thedelivery destination (related construction work group ID, relevantcompany ID, or construction work supervisor ID) identified by thedelivery destination pattern.

Additionally, the delivery processing unit 216 obtains the contactinformation of the identified delivery destination from the participantinformation memory 23 and delivers the inspection result to theparticipant.

Information Displaying Process

The information displaying process will now be described with referenceto FIG. 4. This process is executed if an inspection result is checkedusing the user terminal 20. In this case, the user (construction workgroup, person in charge at construction company, or the like) moves to aposition where the user can photograph the initial position markerattached to a structure. In the user terminal 20, an inspection resultdisplaying program is activated.

In this case, the control unit 21 executes the user authenticationprocess in the same manner as step S1-1 (step S2-1). In this case, theuser authentication unit 210 also identifies the user based on a user IDinput to the login view.

Then, in the same manner as steps S1-2 to S1-4, the control unit 21executes a process for identifying the initial position (step S2-2), aprocess for identifying the current place (step S2-3), and asuperimposition displaying process of the BIM model (step S2-4).

Subsequently, the control unit 21 executes a process for determiningwhether or not an inspection result is included (step S2-5). Morespecifically, the inspection assistance unit 214 of the control unit 21identifies an element ID included in a two-dimensional image. If theinspection management record 250 about the element ID is recorded in theinspection information memory 25 of the inspection management record250, the inspection assistance unit 214 determines that the inspectionresult is included.

If the control unit 21 determines that the inspection result is included(“YES” in step S2-5), the control unit 21 executes a process fordetermining whether or not the user is a related person (step S2-6).More specifically, if the authenticated user ID is recorded in each ofthe data regions of the relevant construction work group ID, therelevant company ID, and the delivery destination ID, the inspectionassistance unit 214 determines that the user is a related person.

If the control unit 21 determines that the user is a related person(“YES” in step S2-6), the control unit 21 executes a process fordisplaying an inspection result (step S2-7). More specifically, theimage output unit 213 outputs an icon including the inspection result toan image displayed on the touch panel display 26. This icon is linked tothe inspection management record 250. If shared information ofindication matters or the like other than the inspection result isrecorded, an icon corresponding to the shared information is displayedon the touch panel display 26 in the same manner. By selecting the icon,the content of the inspection management record 250 can be checked.

In this case, as shown in FIG. 5, the BIM image 510 corresponding to thephotographed image 500 is generated. Further, if the inspectionmanagement record 250 is recorded, the image output unit 213 generates aBIM image 530 including an icon 531 that displays an inspection result.The image output unit 213 generates an output image 540, in which theBIM image 530 is superimposed on the photographed image 500, and outputsthe output image 540 to the touch panel display 26.

If the control unit 21 determines that the inspection result is notincluded or determines that the user is not a related person (“NO” instep S2-5 or step S2-6), the control unit 21 skips the process fordisplaying the inspection result (step S2-7).

In the same manner as step S1-11, the control unit 21 executes a processfor determining whether or not the inspection has ended (step S2-8). Ifthe control unit 21 determines that the inspection has not ended (“NO”in step S2-8), the control unit 21 repeats the processes subsequent tothe process for identifying the current place (step S2-3).

If the control unit 21 determines that the inspection has ended (“YES”in step S2-8), the control unit 21 ends the process for displaying theinspection result.

Measurement Assistance Process of Anchorage Length

The measurement assistance process of the anchorage length will now bedescribed with reference to FIGS. 6A and 6B.

As shown in FIG. 6A, the control unit 21 executes a process foridentifying assembled rebars (step S3-1). More specifically, themeasurement assistance unit 215 identifies an end of a column rebar in aBIM image superimposed on a photographed image. Further, the measurementassistance unit 215 identifies, in the photographed image, an anchoringdevice through image recognition. If an anchoring device cannot beidentified through image recognition, the person in charge of inspectionmay designate an anchoring device on the touch panel display 26.

As shown in FIG. 6B, the measurement assistance unit 215 identifies anend 550 of a column rebar and an anchoring device 551 in an imagephotographed with the camera 28.

Next, the control unit 21 executes a process for calculating ananchorage length (step S3-2). More specifically, the measurementassistance unit 215 calculates the length from the end 550 of the columnrebar to the anchoring device 551 through three-dimensional measurementof the image photographed with the camera 28.

Then, the control unit 21 executes a process for checking the anchoragelength (step S3-3). More specifically, the measurement assistance unit215 compares the calculated anchorage length (measurement value) withthe design value of the inspection item (anchorage length) recorded inthe inspection management record 250 of the inspection informationmemory 25. If the measurement value is within an allowable range set inadvance for the design value, the measurement assistance unit 215records pass as the inspection result. If the measurement value exceedsthe allowable range, the measurement assistance unit 215 records fail asthe inspection result.

Measurement Assistance Process of Pitch and Rebar Diameter

The measurement assistance process of pitch and rebar diameter will nowbe described with reference to FIGS. 7A to 7E.

As shown in FIG. 7A, the control unit 21 executes a process foridentifying assembled rebars (step S4-1). More specifically, themeasurement assistance unit 215 identifies the assembled rebars using aphotographed image (point group data) of a column rebar.

In this case, point group data 560 shown in FIG. 7B is obtained.

Then, the control unit 21 executes a process for calculating a pitch(step S4-2). More specifically, the measurement assistance unit 215 cutsout a predetermined region of the point group data 560 to identify aninspection surface of the column rebar. The measurement assistance unit215 identifies a column main bar and a hoop bar on the inspectionsurface.

As shown in FIG. 7C, on an inspection surface 570, a main bar frame 571is set in a horizontal direction and a hoop bar frame 572 is set in avertical direction as a reference unit length. The main bar frame 571and the hoop bar frame 572 are each arranged to vertically intersect theextension direction of a rebar subject to inspection. Subsequently, themeasurement assistance unit 215 calculates the number of rebars from anaggregation of point group data included in each frame (571, 572). Themeasurement assistance unit 215 calculates a pitch by dividing thereference unit length by the number of rebars.

As shown in FIG. 7D, six rebars (main bars) are detected in the main barframe 571.

Subsequently, the control unit 21 executes a process for calculating arebar diameter (step S4-3). More specifically, in the point group data,the measurement assistance unit 215 calculates a circle on the outeredge of each rebar. The measurement assistance unit 215 calculates therebar diameter from the diameter of the circle.

As shown in FIG. 7E, the measurement assistance unit 215 fits the pointgroup data 581 with rebar circles 582 of D29, D32, and D35 to predicteach rebar diameter.

Afterwards, the control unit 21 executes a process for verifying aresult (step S4-4). More specifically, the measurement assistance unit215 compares the measurement value of the calculated pitch and the rebardiameter with the design value of the inspection item (pitch and rebardiameter) recorded in the inspection management record 250 of theinspection information memory 25. In this case, if the measurement valueis within the allowable range set in advance for the design value, themeasurement assistance unit 215 records pass as the inspection result.If the measurement value exceeds the allowable range, the measurementassistance unit 215 records fail as the inspection result.

The inspection processing system of the present embodiment has theadvantages described below.

(1) In the present embodiment, the control unit 21 executes the processfor identifying the initial position (steps S1-2 and S2-2), the processfor identifying the current place (steps S1-3 and S2-3), and thesuperimposition displaying process of a BIM model (steps S1-4 and S2-4).This allows for inspection using a BIM image corresponding to thecurrent position. In particular, the superimposition of a photographedimage and a BIM image allows for efficient understanding of thedifference between the design and the actual installation.

(2) In the present embodiment, the control unit 21 executes the processfor determining whether or not an element is subject to inspection (stepS1-5). If the control unit 21 determines that the element is subject toinspection (“YES” in step S1-5), the control unit 21 executes themeasurement assistance process (step S1-6). This allows a person incharge of inspection to easily understand inspection points.

(3) In the present embodiment, if the control unit 21 determines that aninspection result has been input (“YES” in step S1-7), the control unit21 executes the process for recording the inspection result (step S1-8).If the control unit 21 determines that there is urgency (“YES” in stepS1-9), the control unit 21 executes the urgent delivery process (stepS1-10). Thus, information about the inspection result having highimportance and urgency can immediately be shared among participants.

(4) In the present embodiment, if the control unit 21 determines thatthe inspection has ended (“YES” in step S1-11), the control unit 21executes the process for delivering the inspection result (step S1-12).This allows information to be shared among participants.

(5) In the present embodiment, if the control unit 21 determines that aninspection result is included (“YES” in step S2-5), the control unit 21executes the process for determining whether or not the user is arelated person (step S2-6). If the control unit 21 determines that theuser is a related person (“YES” in step S2-6), the control unit 21executes the process for displaying the inspection result (step S2-7).This allows the participant (logged-in user) to check the inspectionresult using a photographed image on which a BIM image is superimposed.

(6) In the present embodiment, the control unit 21 executes the processfor identifying assembled rebars (step S3-1), the process forcalculating an anchorage length (step S3-2), and the process forchecking the anchorage length (step S3-3). Thus, the inspection of theanchorage length of a rebar can be assisted using a photographed image.

(7) In the present embodiment, the control unit 21 executes the processfor identifying assembled rebars (step S4-1), the process forcalculating a pitch (step S4-2), the process for calculating a rebardiameter (step S4-3), and the process for verifying a result (stepS4-4). Thus, the inspection of a rebar pitch and a rebar diameter can beassisted using point group data.

It should be apparent to those skilled in the art that the presentdisclosure may be embodied in many other specific forms withoutdeparting from the spirit or scope of the disclosure. Particularly, itshould be understood that the present disclosure may be embodied in thefollowing forms.

In the above embodiment, the management server 10 and the user terminal20 are used. The hardware configuration is not limited to theseelements. For example, data obtained in the user terminal 20 may be usedso that the management server 10 performs various types of informationprocessing and outputs the processing result with the user terminal 20.

In the above embodiment, the user terminal 20 includes the touch paneldisplay 26. The superimposition displaying of a BIM model may beperformed with a head-mounted display (HMD).

In the above embodiment, the current position and orientation of theuser terminal 20 are identified based on acceleration informationobtained from the inertia measurement unit 27. However, the currentposition does not have to be identified in such a manner. For example, aglobal positioning system (GPS) or the technique described in JapaneseLaid-Open Patent Publication No. 2017-45404 may be used.

In the above embodiment, the control unit 21 executes thesuperimposition displaying process of a BIM model (step S1-4). Themethod for displaying a BIM image may be changed based on an evaluationresult of the measurement assistance process. For example, the controlunit 21 changes the color scheme of a BIM image based on a fittingstatus between the BIM image and a structure image. More specifically,the image output unit 213 calculates the difference between a BIM imageand a structure image for each region. The image output unit 213displays the BIM image in a thick, vivid color for a region having alarge difference between the BIM image and the structure image anddisplays the BIM image with a high transparency for a region having asmall difference between the BIM image and the structure image.

In FIG. 8A, a structure image 601 matches a BIM image 602. Thus, the BIMimage 602 is displayed in a thin color. In FIG. 8B, a structure image611 deviates from a BIM image 612. Thus, the color of the BIM image 612is changed and displayed thick in a lower region, where the structureimage 611 greatly deviates from the BIM image 602.

In the above embodiment, the control unit 21 executes thesuperimposition displaying process of a BIM model (step S1-4). In thiscase, a virtual image set at a predetermined transmittance is used as aBIM image. Instead, the image output unit 213 may extract the contour ofa three-dimensional model of an element and use a virtual image with aframe model.

In the above embodiment, the control unit 21 executes the process forrecording an inspection result (step S1-8). In this case, the person incharge of inspection inputs, to the information registration view, aninspection item determination result including importance and urgency.Instead, the inspection assistance unit 214 of the control unit 21 mayassist determination of importance and urgency. For example, theinspection assistance unit 214 determines importance and urgency usingthe content of an inspection item and attribute information of athree-dimensional model recorded in the BIM information memory 22. Inthis case, if the difference between the design value and themeasurement value is large, the inspection assistance unit 214 increasesthe importance. Further, weighting may be performed for importance inaccordance with an inspection item. For example, weighting is set to belarge for an element that is difficult to deal with and an inspectionitem that would greatly affect other constructions.

Further, importance and urgency may be determined in accordance with aconstruction schedule. In this case, for example, the user terminal 20includes a schedule memory that records the schedule of eachconstruction. Further, the control unit 21 causes the user terminal 20to keep a dealing period table that is used to calculate a time requiredfor dealing in accordance with the inspection result (for example,difference between design value and measurement value) of eachinspection item. In the inspection assistance process, the inspectionassistance unit 214 obtains, from the schedule memory, the schedule(scheduled date) of a subsequent construction in which elements subjectto inspection are affected. In addition, the inspection assistance unit214 obtains a required time corresponding to the inspection result fromthe dealing period table. The inspection assistance unit 214 determinesurgency based on the comparison of a correction scheduled date, which isobtained by adding a required time to the current date, with asubsequent construction scheduled date. The inspection assistance unit214 calculates a high urgency if the correction scheduled date is closeto the subsequent construction scheduled date or the correctionscheduled date passes the subsequent construction scheduled date.

In the above embodiment, the control unit 21 executes thesuperimposition displaying process of a BIM model (step S1-4). Thecontrol unit 21 generates the output image 520, in which the BIM image510 is superimposed on the photographed image 500, and outputs theoutput image 520 to the touch panel display 26. The superimpositiondisplaying process is used for the inspection assistance process and themeasurement assistance process of a rebar and the like. The inspectionsubject is not limited to a rebar.

For example, the superimposition displaying process may be applied tomeasurement and surveying of pile centering or the like. In this case,the control unit 21 causes the user terminal 20 to keep pile centeringlayout information. The control unit 21 generates a pile core or abaseline corresponding to the pile centering layout as athree-dimensional model of a structure. Then, the control unit 21 usesthe three-dimensional model to generate a virtual image corresponding tothe current position and superimpose the virtual image on a photographedimage.

Further, the superimposition displaying process may be applied tomeasurement and surveying of plumbing or the like of a steel frame orprecast concrete (PCa) element. In this case, the control unit 21 uses aBIM model of a steel frame or PCa as a three-dimensional model of astructure to generate a virtual image corresponding to the currentposition and superimpose the virtual image on a photographed image.

In addition, the superimposition displaying process may be applied tointerference checking of a seismic reinforcement iron frame and facilityand interference checking of a scaffold and skeleton. In this case, thecontrol unit 21 uses a BIM model of a seismic reinforcement iron frameand facility and a BIM model of a scaffold and skeleton as athree-dimensional model of a structure to generate a virtual imagecorresponding to the current position and superimpose the virtual imageon a photographed image.

Thus, the superimposition displaying process may be applied tosurveying, measurement, inspection, and the like using a superimposedimage of an invisible part that cannot be seen in the real space (forexample, design) and a visible part that can be seen (for example,current state).

The control unit 21 is not limited to a device that performs a softwareprocess for all the processes executed by the control unit 21. Forexample, the control unit 21 may include a dedicated hardware circuit(for example, application specific integrated circuit (ASIC)) thatperforms a hardware process for at least part of the processes executedby the control unit 21. That is, the control unit 21 may be configuredby circuitry including (1) one or more processors running on computerprograms (software), (2) one or more dedicated hardware circuits thatexecute at least part of various types of processes, or (3) acombination thereof. The processor includes a CPU and a memory such as aROM or a RAM, which store programs executed by the CPU. The memory, orcomputer readable medium, includes any type of medium that is accessibleby a versatile computer or dedicated computer.

Therefore, the present examples and embodiments are to be considered asillustrative and not restrictive and the disclosure is not to be limitedto the details given herein, but may be modified within the scope andequivalence of the appended claims.

1. An inspection processing system comprising circuitry, wherein thecircuitry is configured to execute: a registration process to identify acurrent position, obtain an inspection result of an inspection subjectof a structure on the current position, and record the inspection resultin association with the position of the inspection subject in athree-dimensional model of the structure; and a displaying process toidentify a user who views the three-dimensional model and a currentposition, and output, to an output unit, an inspection result in whichthe user is recorded in an inspection information memory as a relatedperson of the inspection subject, wherein the inspection result isattached on a virtual image according to a three-dimensional modelcorresponding to the current position.
 2. The inspection processingsystem according to claim 1, wherein the circuitry is configured toidentify a visual range of a photographing unit that photographs thestructure, and output the inspection result of the inspection subjectincluded in the visual range in the three-dimensional model.
 3. Theinspection processing system according to claim 1, wherein the circuitryis configured to identify a related person who shares information basedon constructor information of the structure.
 4. The inspectionprocessing system according to claim 1, wherein the circuitry isconfigured to identify a delivery level of the inspection result, andidentify a related person who shares information based on the deliverylevel.
 5. The inspection processing system according to claim 1, whereinthe circuitry is configured to identify a construction schedule of theinspection subject, and identify a related person who shares informationbased on the construction schedule.
 6. A method for assisting inspectionusing an inspection processing system comprising: by circuitry of theinspection processing system, executing a registration process toidentify a current position, obtain an inspection result of aninspection subject of a structure on the current position, and recordthe inspection result in association with the position of the inspectionsubject in a three-dimensional model of the structure; and by thecircuitry, executing a displaying process to identify a user who viewsthe three-dimensional model and a current position, and output, to anoutput unit, an inspection result in which the user is recorded in aninspection information memory as a related person of the inspectionsubject, wherein the inspection result is attached on a virtual imageaccording to a three-dimensional model corresponding to the currentposition.
 7. The method according to claim 6, the method furthercomprising: by the circuitry, identifying a visual range of aphotographing unit that photographs the structure, and by the circuitry,outputting the inspection result of the inspection subject included inthe visual range in the three-dimensional model.
 8. The method accordingto claim 6, the method further comprising: by the circuitry, identifyinga related person who shares information based on constructor informationof the structure.
 9. The method according to claim 6, the method furthercomprising: by the circuitry, identifying a delivery level of theinspection result, and by the circuitry, identifying a related personwho shares information based on the delivery level.
 10. The methodaccording to claim 6, the method further comprising: by the circuitry,identifying a construction schedule of the inspection subject, and bythe circuitry, identifying a related person who shares information basedon the construction schedule.
 11. A non-transitory computer-readablemedium that stores a program thereon, wherein the program, when executedby circuitry of an inspection processing system, causes the circuitry toexecute: a registration process to identify a current position, obtainan inspection result of an inspection subject of a structure on thecurrent position, and record the inspection result in association withthe position of the inspection subject in a three-dimensional model ofthe structure; and a displaying process to identify a user who views thethree-dimensional model and a current position, and output, to an outputunit, an inspection result in which the user is recorded in aninspection information memory as a related person of the inspectionsubject, wherein the inspection result is attached on a virtual imageaccording to a three-dimensional model corresponding to the currentposition.
 12. The non-transitory computer-readable medium according toclaim 11, wherein the program, when executed by the circuitry, causesthe circuitry to identify a visual range of a photographing unit thatphotographs the structure, and output the inspection result of theinspection subject included in the visual range in the three-dimensionalmodel.
 13. The non-transitory computer-readable medium according toclaim 11, wherein the program, when executed by the circuitry, causesthe circuitry to identify a related person who shares information basedon constructor information of the structure.
 14. The non-transitorycomputer-readable medium according to claim 11, wherein the program,when executed by the circuitry, causes the circuitry to identify adelivery level of the inspection result, and identify a related personwho shares information based on the delivery level.
 15. Thenon-transitory computer-readable medium according to claim 11, whereinthe program, when executed by the circuitry, causes the circuitry toidentify a construction schedule of the inspection subject, and identifya related person who shares information based on the constructionschedule.